Cluster-Computing and Parallelization for the Multi-Dimensional PH-Index Master Thesis

The storage and management of multi-dimensional data is an important aspect in many applications, like geo-information systems, computer vision and computer geometry. In contemporary times, when computers are able to capture and store increasing amounts of multidimensional data, it is important for applications to efficiently store and query this data. This work presents a distributed version of the PH-tree, a highly efficient in-memory multidimensional data structure, supporting range and nearest neighbour queries. We present a distribution architecture for the PH-tree, which extends it to run on a cluster of computers. The distributed version of the PH-tree is able to use the main-memory of all of machines in the cluster to store the multi-dimensional data. Moreover, the distributed setting allows each machine to be queried independently of the other machines. The point distribution algorithm proposed in this work uses the Z-order space filling curve to assign sections of the space to the computers in the cluster. Additionally, we present an automatic data rebalancing algorithm, which attempts to maintain an equal storage load across all computers. The performance evaluation shows that the proposed distribution version of the PH-tree is able to scale with respect to the number of computer in the cluster, obtaining an almost linear increase in throughput and a similar reduction in the response time for point operations. Another contribution of this work is the extension of the PH-tree to support concurrent write access, allowing it to take advantage of multi-core processor architectures. We present several concurrent write access strategies, with different consistency guarantees, discuss their advantages and disadvantages and evaluate the write performance. The presented copy-onwrite strategy allows queries to execute on snapshots of the PH-tree, while the PH-tree itself is accessed by one writer and multiple readers. This work also presents two fine grained locking strategies, which sacrifice consistency to allow multiple reader and writer threads to access the tree in the same time. All presented concurrency strategies perform locking only in the case of the write operations, and allow read queries to execute without taking any locks, which greatly increases the performance of read operations for large numbers of reader threads.